Consistency Regularization for Adversarial Robustness

Adversarial training (AT) is currently one of the most successful methods to obtain adversarial robustness deep neural networks. However, phenomenon robust overfitting, i.e., starts decrease significantly during AT, has been problematic, not only making practitioners consider a bag tricks for training, e.g., early stopping, but also incurring significant generalization gap in robustness. In this paper, we propose an effective regularization technique that prevents overfitting by optimizing auxiliary `consistency' loss AT. Specifically, discover data augmentation quite tool mitigate and develop forces predictive distributions after attacking from two different augmentations same instance be similar with each other. Our experimental results demonstrate such simple brings improvements test accuracy wide range AT methods. More remarkably, show our method could help model generalize its against unseen adversaries, other types or larger perturbations compared those used training. Code available at https://github.com/alinlab/consistency-adversarial.